Content and Link Structure Analysis for Searching the Web

Automated search engines continuously discover, index, and store information about web pages. When a user issues a query, this repository is searched to find a result set of most relevant pages. An ideal search scheme must satisfy two basic requirements: high recall, and high precision. Recall measures the ability of an algorithm to find as many relevant pages as possible. Precision measures the ability of an algorithm to reject as many nonrelevant pages as possible. An ideal search algorithm should find all of the relevant pages, rank them by relevance to the user query, and present a rank-ordered result to the user. The earlier generations of search engines relied solely on keyword matching to perform the search. Unfortunately this approach didn’t work very well. Too many nonrelevant pages were returned along with relevant ones, and their rankings rarely agreed with users’ interests. Since user queries are short, usually consist of 2-3 words, the problems associated with synonyny and polysemy make it particularly difficult to evaluate which pages will be of interest to a user. The user is more likely to be interested in a page if it contains authoritative information on its subject and it is relevant to the user query. Authoritative pages are usually cited by others frequently, and the link